U.S. PATENT APPLICATION 
INVENTORS: HIDEO NORO, ET AL. 
TITLE: IMAGE PROCESSING METHOD AND 
P205-015 8 WO IMAGE PROCESSING APPARATUS 

Sheet 2 of 6 

Docket No.: 00862.109670 



2/6 




U.S. PATENT APPLICATION 
INVENTORS: HIDEO NORO, ET AL. 
TITLE: IMAGE PROCESSING METHOD AND 
P205-0158WO IMAGE PROCESSING APPARATUS 

Sheet 3 of 6 

Docket No.: 00862.109670 



3/6 



DC 
co 
o 



LU OC 

LU 
CD 



CO 
O 

H 



O LU 



LU: 



eg 
o 



DC 
c\j 
o 

SI 



o 
o 



-^LU 

cd >- 

UJ^LLJ 
CL.CC 

o 
o 



DC 
O 



LU l3 I — 

8§ 



LULU 
DC >- 
ZD LU 

LU £ 



o 

SI 



DC 

o 

^LU 

5c lu 

Qh- 

o& 

OZj 



DC 

^ LU 

on: 

O DC 



LO 
O 

^1- 



LUj 
DC 1 



LU^ 
CDt= 

:< ^ 




A" 



CO 

o 



<r LU 
pb CO 

Q O CD 
^ Q 

O Q 



Qbo 
coi 



o 



O 

SI 



O — CD 
rS^CO 

FRdcco 

— DC 



o 



oy=!oo_ 

O-DCLU^ 



o.z 
F=luQ 

yj 2: oc 
o -z. 

LU 
<2- 







OS 


LUS 


H- LU 


Z CO 


Is 


J? LU 




LU LU 


Qui 


ON/01 
RING 


ERAT 

LAYD 


OSITI 
EASU 


OOP 
DISPI 




1— 



8 

CO 
LU 
O 

> 
LU 
Q 



O 
DC 
LL_ 



CO 

d 



U.S. PATENT APPLICATION 
INVENTORS: HIDEO NORO, ET AL. 
one Icoha TITLE: IMAGE PROCESSING METHOD AND 

P205-0158WO IMAGE PROCESSING APPARATUS 

Sheet 4 of 6 

Docket No.: 00862.109670 



4/6 



FIG. 4 



r 4010 

MEASURE 
POSITION AND 
ORIENTATION OF 
OBJECT MANIPUL 
ATION DEVICE 



4020 



MEASURE POSITION 
AND ORIENTATION OF 
3D POINTING DEVICE 



4040 




MEASURE POSITION 
AND ORIENTATION 
OF HEAD 



4030 



CALCULATE POSITION 
AND ORIENTATION OF 
CORRESPONDING 
CG OBJECT 




CALCULATE POSITION 
AND ORIENTATION 
OF CG CAMERA 




3D CG SCENE 
GRAPH DATABASE 



-406 



4070 

__L_ 



INPUT PHYSICAL 
SPACE IMAGE 



4110 

s 



ACQUIRE 
OPERATION 
INFORMATION 




PROCESS 
OPERATION 
INFORMATION 



RENDER CG 



4080 



RENDER 
PHYSICAL 
SPACE 



4130 



GENERATE 
OPERATION PANEL 



4140 




4090 



SUPERIMPOSE 
IMAGE INFORMATION 



OUTPUT 
OPERATION PANEL 



11 



4100 



OUTPUT MIXED 
REALITY IMAGE 



U.S. PATENT APPLICATION 
INVENTORS: HIDEO NORO, ET AL. 
TITLE: IMAGE PROCESSING METHOD AND 
P205-0158WO IMAGE PROCESSING APPARATUS 

Sheet 5 of 6 

Docket No.: 00862.109670 



5/6 



X 






|d 






[' 




rr 






<c 

LU 






CD 












M 






CD 
CD 






CD 






CM 










'OOL(T) 


o 

i 




i — 


co 




LU 

I— 

Q 
LU 






CIT 

LU 
1 


LO 



O 

LO 



CO 

ID 

CO 



DC 
O 

<c 

cr <r 

LLI H— 
=J O 

2 ^ 

DC Q_ 

o o 

DC 
LU 



O 
Q_ 

DC 
LU 

© O 











































No.2 


No.1 


X1.0000 




CO 

< 

CQ 
Q 
O 
co DC 

co O 



GC 
O 

><o ^£ 



CO 



Q°-| 

I CD 

cp o 

CO CO 
CQ CQ 
DC DC 
^t- to 



3^. 

O CO 

o o 
i i 

CO CO 
CQ CQ 
DC DC 



CO 
O 



§Sco^ 
x 

X 

oo o o 

CO CM CO CM 

oo oo 



oo 



sag 



I o I o I o| o I oj o [ ct, fol o| o| o| < 



c\j ■ . . 

t- N- C\J CVJ 
CQ CQ 
DC DC 

LO 

o 

I 21 

CD 



DC 



o ( o 

CO Tj- 

o o 
^ i^- 

CO CO 
DC DC 
t-CM CO^I- 



5J[J1-J 

QJ-J 



EH" 
m 



Is 



DC QC 

<c 

LU LU 

CQ CQ 

CD CD 



LU LU 
1 — I — 

CO CO 

<c <c 

LJL LJL 

o o 
co 



DC 

!<=>' 

S CM 
i O 
LU I 

§ 5 , 

LO ^ 
<P CO 



OO 



X 

3^3 

LO P CO 

S S2 

CO o o 

c\j oS 
CQ co o 
DC DC X 

O i— CM 
CM CM CM 



00 



□ 



o 
o 

LO 



<M 
O 
LO 



LO 
O 
LO 



o 

LO 



in 



(5 

LL 



U.S. PATENT APPLICATION 
INVENTORS: HIDEO NORO, ET AL. 
TITLE: IMAGE PROCESSING METHOD AND 
P205-0158WO IMAGE PROCESSING APPARATUS 

Sheet 6 of 6 

Docket No.: 00862.109670 



6/6 



FIG. 6 





EXTERNAL 






STORAGE DEVICE 





601- 



602- 



603- 





CPU 









RAM 








ROM 







604- 



605- 



OPERATION 




UNIT 





DISPLAY UNIT 



606 



607 



l/F 



EXTERNAL 
DEVICE 



